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ABSTRACT 

The development and structure of the Profile of 
School Excellence (PRO-S/E) are described, and the needs for which 
the PRO-S/E was developed are explained. A product of the Appalachia 
Educational Laboratory (AEL) in West Virginia, the PRO-S/E has been 
involved in evaluation and improvement planning and intervention for 
25 urban and regional school districts. The AEL offers School 
Excellence Site status to districts within AEL's service region 
(Kentucky, Tennessee, Virginia, and West Virginia) following their 
participation in the PRO-S/E process. Most schools call on the AEL 
for follow-up technical assistance after PRO-S/E studies. Assistance 
usually involves the provision of inservice training. In 1985, the 
PRO-S/E teacher and student surveys (for grades 1-4 and 5-12) were 
revised. In addition, a factor analysis of the four PRO-S/E 
instruments was undertaken, and the data analysis system, which now 
runs on a BASIC language program, was upgraded. Brief analyses are 
provided of the use of PRO-S/2 in a school district in Lexington, 
Kentucky; training of university faculty in Kentucky, Tennessee, and 
West Virginia to use the PRO-S/E; and follow-up with school districts 
after use of the PRO-S/E. Five figures are attached. (TJH) 
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DEVELOPMENT OF THE PROFILE OF SCHOOL EXCELLENCE - Jack Sanders 



The purpose of this section of the paper is to describe the development of 
the Profile of School Excellence (PRO-S/E). including an explanation of the 
needs for which the PRO-S/E was developed, an explanation of the literature 
on which the PRO-S/E was based, a review of the structure of the PRO-S/E. the 
use of the PRO-S/E by practitioners, and suggestions for next steps in its 
continued development. 

Need and Rationa le 

The PRO-S/E was developed in response to the expressed needs of local 
school superintendents, principals, board members, and supervisors in AEL's 
service region. This developir^ent began in 1982. Prior to that time, school 
administrators had been barraged with information about the outcomes of re- 
search on effective schools. The information came from presentations at pro- 
fessional meetings and journal articles, as well as from workshops and publi- 
cations provided by AEL and other service providers. Also, these administra- 
tors were sensitized to the spate of forthcoming national education status 
reports from groups such as the Business-Higher Education Forum, the National 
Commission on Excellence in Education, the Paideia Group, and the Twentieth 
Century Fund Task Force on Federal Elementary and Secondary Policy; from 
organizations such as the College Board. Education Commission of the States, 
the National Association of Independent Schools, the National Association of 
Secondary School Principals, and the National Center for Educational Statis- 
tics; and from such Individuals as Ernest Boyer, James Coleman, Emily 
Felstrltzer, and John Goodlad. The administrators In the region asked AEL to 
develop a method for them to respond rationally to this new Information about 
effective schools. The method, they said, should provide them with a basis 
for discussing with teachers, staff, and board members possible alternatives 
about the allocation of school Improvement resources. The administrators, 
many from rural and resource-poor districts, urged that the method be In- 
expensive and non-disruptive of school routines. Finally, school officials 
urged that the method produce a report which presented outcomes understand- 
ably, using both narrative and graphic formats. 

Resource Review 

AEL responded by searching for extant Instruments and processes that might 
address or be adapted to address the needs and conditions articulated by the 
actnlnlstrators. Literature searches and communications with the network of 
Regional Laboratories and unl verslty-based R&D Centers produced literature 
synthesis and lists of school effectiveness charaterl sties, but no validated 
instruments or processes that could be adapted and packaged to address the 
need. Of the syntheses and lists, two were judged pertinent to the AEL de- 
velopment effort: <1) a synthesis of 11 characteristics of effective schools 
and related literature by Larry Hutchlns of McREL, and (2) the five correl- 
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ates of effective schools reported by the late Ron Ednonds. Sensitive to the 
fact that many of the early school effectiveness studies were conducted in 
urban elementary settings. AEL developers elaborated the Hutchins literature 
review and sharpened the definitions of the 11 characteristics: needs basis, 
objectives, expectations, roles and responsibilities, conditions and resour- 
ces. Instructional time and task orientation, use of assessment, rewards and 
reinforcement, code of behavior, school climate, and parental involvement 
(see Attachment A for definitions). From Ecbionds'' five correlates. AEL adap- 
ted the notion of disaggregating standardized test scores as a twelfth 
Indicator of effectiveness. Disaggregation allows for comparison of high and 
low SES group attainment of basic skills. The characteristics and the devel- 
oper ''s adaptations are described further in Sanders, Shively, and Machesney 
(<984> and Sanders, Barnette. and Vanco (1986). 

Structure 

To assess the characteristics. AEL developed eight data collection instr- 
ments for use in local school districts (see Attachment B for further details 
on these instruments). Prior to an on-site visit by PRO-S/E administrators, 
district personnel receive and complete four types of instruments: district 
data form, superintendent interview form, school data form (one for each 
school), and principal interview form (one for each principal). During the 
site visit, the central office staff familiar with the schools complete 
school rating forms for each of the schools, interviews are conducted with 
the superintendent and principals, and other instruments are left for the 
principal to administer in his/her school: a teacher survey (50% sample), 
student surveys for grades 1-4 and 5-12 (10% sainpie), and a form for report- 
ing disaggregated test scores. Within five days, all surveys are to be 
administered and returned to AEL for analysis and interpretation. 
Shortly thereafter, the disaggregated test scores are submitted to AEL for 
analysis. AEL prepares a written report to the superintendent within ten 
weeks of receipt of all data. The report provides extensive profile and 
suirmary data tables for each school and a narrative report summarizing the 
findings for each characteristic across schools. R & D-based programs ad- 
dressing the most salient needs are identified for school officials^ consid- 
eration. AEL staff provide follow-up technical assistance to the district 
as needed. 

Results 

PRO-S/E studies have been conducted in more than 25 school districts. 
Some of the districts have been urban, some rural, some large, and some 
small. Officials of most districts using the PRO-S/E report that the study 
was used as a basis for subsequent school improvement planning and interven- 
tion. Officials also report that PRO-S/E study results confirm their per- 
ceptions about the strengths and weaknesses of the districts schools. Most 
districts call on AEL for follow-up technical assistance after PRO-S/E stu- 
dies. This assistance typically involves provision of inservice training 
aimed at addressing areas diagnosed as needing attention. 
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Next Steps 

The PRO-S/E was developed as a diagnostic tool school districts could use 
as a basis for school irm^rovement planning. In addition to the factor analy- 
tic studies about the PRO-S/E characteristics which are presently being con- 
ducted, AEL is interested in surveying prior adoptation sites to see if 
changes occurred and the types of changes after use of the PRO-S/E in the 
district. We would like to know if the district has used other diagnostic 
assessments which produced findings similar or contradictory to PRO-S/E 
findings. We would like to know if the district intends to repeat the use 
of the PRO-S/E, as some have, to see if perceptions about local conditions 
have changed, particularly those conditions identified as in need of atten- 
tion to Virtiich remediation efforts were directed. Finally, we want to reanal- 
yze both the literature and practitioner experience to identify any special 
school effectiveness correlates vrfiich should be considered as additions or 
substitutes when studying effectiveness in rural/isolated school districts. 



PRO-S/E DATA ANALYSIS SYSTEM - Jack Barnette 

The purpose of this section is to describe three activities which were or 
are being conducted to ensure that the instruments possess content/construct 
validity and the data analysis system was easy to use and provided useful 
output to PRO-S/E report writers and school district personnel. 

Instrument Revisiion 

In 1985, the PRO-S/E teacher and student surveys were revised. It had 
been recognized by the PRO-S/E developer"* thet some of the items were not 
clear, some were asking more than one thing, and others, which used a •'check 
all that apply" format, created some problems in scoring and analysis. All 
of the items were reviewed on these instruments and several were revised to 
eliminate or minimize problems in interpretation and scoring. The revised 
set of items was reviewed by five persons who were very familiar with the 
school effectiveness literature. These reviewers had two tasks. One was to 
assess the clarity of each item and the appropriateness of the response set 
which accompanied the item. The other was to categorize the item into one 
of the 11 characteristics. Very few of the items were considered in need of 
revision relative to wording or response set. Items were placed into the 
11 characteristics based on the consistency of categorization agreement of 
the reviewers. A few items were dropped due to lack of agreement and a few 
others were modified slightly to fit more closely with the characteristic 
definitions. The reviewers were: Lynn Canady, Phyllis Hotchkiss, Tan 
Bennett, Joanne Reina, and Jean Coolican. 

Prior to 1986, the student survey instrument was given to students in 
grades 5 or al30ve. It was determined that it mig^it be useful to have a 
student survey for students in the lower grades. This instrument was devel- 
oped and field testing in three schools in Memphis, TN in April 1986. The 
instrument instructions included procedures for reading the items to students 
in grades 1 and 2, and having students in grades 3 and 4 reading the instru- 
ment themselves. Also, prior to this time the staff rating was comprised of 
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a set of 11 items, one for each characteristic, on which the rater rated the 
school on a 1-10 point scale. It was determined that this did not provide 
very much information on why a rating might be high or low, so the instrument 
was expanded to request responses on three to five items within each charac- 
teristic category. These items were developed to parallel items on the 
teacher and student surveys. This instrument was field tested in April 1986. 
Both of these have oeen added to the set of PRO-S/E instruments. 

Factor Analysis Study 

As another approach to testing the validity of the PRO-S/E instruments, 
the author is conducting a factor analysis of the four survey instruments. 
While it has never been the claim that the 11 characteristics are "factors'* 
in the psychometric sense, it was decided that such an analysis could provide 
further information upon which the instruments could be improved or defended 
as having a reasonable level of construct validity. The final report for the 
factor analysis study will not be complete until May 1987. 

While the final analysis has not been completed, it is possible to present 
some preliminary results for the instruments: 

1 . Teacher survey 

Responses from 575 teacher surveys from two school districts were 
factor arvalyzed using a principal components analysis and varimax 
rotation on eleven factors, accounting for 45% of the variance. Fac- 
tor U which accounted for 18% of the variance, was made up primarily 
of items from the needs basis, objectives, conditions/resources, and 
school climate characteristics. Factor 2 had items frcxn several char- 
acteristics, but tended to have more items from the parental involve- 
ment and rewards/reinforcement characteristics. Factor 3 tended to be 
comprised of items from the parental involvement, roles/responsibil- 
ities, and school climate characteristics. Factor 4 tended to be made 
up of items from the expectations and rewards/reinforcement character- 
istics. Factors 5 and 10 were made up totally from items from the 
instructional time/task orientation characteristic. Factor 6 was made 
up of items frcMn the conditions/resources characteristic, factor 7 was 
made up of items from the roles/responsibilities characteristic, fac- 
tor 8 was made up of items from the needs basis characteristic, and 
factor 11 was made up of items from the code of behavior characteris- 
tic. Factor 9 had no majority of items from any characteristic. 

2. Student survey (grades 1-4) 

Responses from 623 surveys from two school districts were factor anal- 
yzed using a principal components analysis and varimax rotation on 
seven factors, accounting for 42% of the variance. The results indi- 
cate that there is not a very clear factor structure relative to the 
eleven characteristics. Use of this instrument has revealed that 
there is relatively low variance in the responses. Elementary stu- 
dents tend to give very positive ratings on this instrument. Only the 
items frcOT roles/responsibilities, needs basis, rewards/reinforcement. 
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and use of assessment tended to load on specific factors. 



3. Student survey (grades 5-12) 

Responses from 1105 surveys from two school districts were factor 
analyzed using a principal components analysis and varimax rotation 
on eight factors, accounting for 35% of the variance. The results in- 
dicate that factor 1 was primarily made up of items from the needs 
basis, rewards and reinforcenent, and about half of the school 
climate characteristic it^ms. Factor 2 was comprised mostly of items 
from the objectives and expectations characteristics. Factor 3 was 
a combination of items from the school climate, conditions and 
resources, and instructional time/task orientation characteristics. 
Factor 4 was clearly comprised of items from the roles and responsi- 
bilities characteristic, factor 5 was clearly comprised of items from 
the parental involvement characteristic, and factor 6 was clearly 
con?>rised of items from the code of behavior characteristic. Factor 
7 tended to re 'ate to items from the use of assessment characteristic 
and factor 8 tended to have items relating to the instructional time/ 
task orientation characteristic. While this factor structure is not 
totally consistent with the separate 11 characteristics, the results 
are somewhat supportive of the set of characteristics in that those 
factors which have items from different characteristics could reason-- 
ably expect to have had the combinations observed. 

4. Staff rating form 

The responses on the staff rating form for 351 respondents from two 
school districts were factor analyzed using a principal components 
analysis and varimax rotation on nine factors, accounting for 58% of 
the variance. The results indicate that for six of the characteris- 
tics item loadings were consistent with factors. These six character- 
istics were: neeas basis (factor 2), objectives (factor 6), instruc- 
tional time and task orientation (factor 7), rewards and reinforce- 
ment (factor 5), code of behavior (factor 3), and school climate 
(factor 1). For some of the other characteristics, all items but one 
were related to a specific factor. These were: expectations (factor 
1), roles and responsibilities (factor 4), use of assessment (factor 
8), and parental support and involvement (factor 1). Note that factor 
1 includes the characteristics of school climate, expectations, and 
parental support and involvement. The only characteristic which had 
more than one item loading on different factors was conditions and 
resources. Two of the items loaded on factor 4, which related to 
roles and responsibilities. The other three items loaded on: factor 
8, related to use of assessment; factor 1, related to school climate; 
and factor 2, related to needs basis. In summary, the staff rating 
form items demonstrated a relatively high level of structure consis- 
tent with the 11 characteristics. 
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Data Analysis System 

The original dat-. analysis system, used prior to 1985, was very time con- 
suming and not as useful as possible. It required manual data entry and 
several steps in the processing to complete the graphic prorile display. 
This author was hired as a consultant to AEL and then as a full-time staff 
member to develop a system which was easier to use and more thorous^i in pro- 
viding potentially useful output. The first fully automated system was de- 
veloped using an Apple computer and card reader for aata input. This system 
was found to be easy to use and it did provide analysis output not available 
previously such as profiles across schools on each characteristic and item 
analysis by characteristic to be used to determine the relationships of items 
to the characteristic score. The major problem with the system was the dif- 
ficulty in using the card reader. Mechanical problems necessitated almost 
constant attention oi the user. With this problem, as well as the need to 
include in the system the two new instruments, it was decided to set up the 
system using an IBM PC and an optical scan reader for data input. This sys- 
tem is now operational and requires a minimum of training or attention for 
its use. The present system is programmed in BASIC and compiled to decrease 
run time. 

The system provides several types of output for use by report writers and 
PRO-S/E clients. Some examples of the output are included as Attachments. 

1. For each school, for total elementary, for total middle, for total 
secondary, and for total district the following are provided: 

a. Characteristic means and standard deviations for each respon- 
dent type: students, teachers, and staff ^Attachment C), 

b. Figure displaying means for each characteristic for each 
respondent type (Attachment D), 

c. Ranking from high to low of the 11 characteristics for each 
respondent type, 

d. Discrepancy analysis for each characteristic among the 
respondent types, 

e. Item analysis for the teacher survey (the item analysis ranks 
the 11 characteristics from high to low and, within each char- 
acteristic, ranks the items high to low on the item scale 
(Attachment E),^ 

f. Item analysis for the student survey (grades 5-12), 

g. Item analysis for the student survey (grades 1-4), and 

h. Item analysis for the staff rating forms. 

2. Means on each characteristic by respondent type are presented across 
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the schools in the district in tabular and graphics forms (Attachments 
F and G). 

3. Achievement by SES group analyais for schools submitting the disaggre- 
gated data, including a test for homogeneity of proportions. 

The analysis outputs permit looking at the data across dimensions of school 
type, characteristic, and respondent type. They provide the basic informa- 
tion needed to interpret several aspects of the PRO-S/E. 



USE OF THE PRO-S/E IN A LOCAL SCHOOL DISTRICT - Allan Osborne 

Having held administrative posts in three school districts in a six year 
span, I had first hand experience with the "terrible T-'s" - turf, tradition, 
and trust. During this period of continued transition, it became apparent 
that - be it fact or fiction - perception is ninety-five percent reality; in 
other words, we are what we think we are. Consequently, upon accepting the 
super intendency in Frankfort, Kentucky's capital, I realized the necessity of 
measuring the prevailing perceptions of the school district. 

Early fall of 1983, I was introduced to the PRO-S/E by Dr. Jack Sanders 
of AEL at an educational consortium meeting in Lexington, Kentucky. Using an 
old cliche, "it was just what the doctor ordered." The PRO-S/E, a research- 
based set of instruments, would allow the board of education* administrative 
staff, faculty, and me to view the district's perceived strengths and weak- 
nesses. This bird'^s eye perceptual view would be that of the students, fac- 
ulty, and administration. The information gained could (eventually would) 
build a stable educational institution. 

The PRO-S/E was actoiinistered on December 2, 1983, and a final report was 
presented to our board of education in late January 1984. At this point, no 
one could imagine what would take place in our small inner-city school dis- 
trict over the next twenty-four months. 

Certain things became evident to us immediately. As practitioners, we 
needed research-based support data to render sound educational decisions; the 
PRO-S/E provided this. Also, it stood to reason that we needed to pay atten- 
tion to research-based solutions. The PRO-S/E final report provided re- 
search-based options which might help in attacking our perceived weaknesses. 

Frankly, from a personal point of view, the most important aspect of the 
PRO-S/E was the protection it offered my hind-side. I had already noted most 
of the perceived weaknesses illustrated by the PRO-S/E, but now I had support 
data which would allow us to direct attention to the perceived weaknesses. 
It was or would not be "Osborne feels and says thus and so." The PRO-S/E 
provided the spring board by which ".;e could dive head first into our problems 
with no one individual or group being the fall guy. 

The entire process was beneficial to the district; however, the research- 
based solutions recommended for our perceived weaknesses proved to be the 
most helpful. Many positive changes have occurred in our district becau.se of 
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the information generated by the PRO-S/E, 

According to the data, our teachers viewed the general area of code of be- 
havior as unacceptable. Quite frankly, it was the general perception that 
the students were in control of the schools and there was general disregard 
for rules and regulations, AEL recommended for our consideration a program 
called Classroom Organization for Effective Teaching (COET). This program 
became an integral part of the 1984-85 inservice program. Staff assistance 
was provided by AEL not only to implement COET, but also to incorporate a 
task management analysis model. 

The most critical weakness as illustrated by the data was the negative 
attitude a irajority of our teachers held concerning the ability of their 
students to achieve the academic objectives of their classes. Simply put, 
teachers had low expectancy for student achievement. The reconmendation here 
was a program known as Teacher Expectations and Student Achievement (TESA), 
Contrary to recommendations, we implemented this program by mandate. TESA, 
more than any single inserv ice/staff development program, has had a positive 
impact on our school district. The Kentucky Department of Education recently 
started a process which ultimately will implement TESA in every district in 
the Commonwealth. The first group of state trainers used our district for 
their observation training. 

The PRO-S/E and AEL share in our many successes of the past three and one- 
half years. The list does not end with COET and TESA but includes CAMEL, 
Talents Unlimited, Adaptive Learning Environmental Model (ALEM), middle 
school reorganization, high school eight period day reorganization, TESA ex- 
pansion model, and more. This is a tribute to AEL and our fifty-eight build- 
ing staff members. ' 

I will be recommending to our board of education that during the 1988-89 
school year we once again utilize the PRO-S/E. For continued success we 
must, on a periodic basis, check our perceptions. Assuredly, we become what 
we think we are. and we are Frankfort Independent Schools. 



TRAINING OF UNIVERSITY FACULTY IN THE USE OF THE PRO-S/E - Peggy Vanco and 
Jane Hange 

The Appalachia Educational Laboratory has a small number of staff members 
trained in conducting, analyzing data, writing, and follow-up of PRO-S/E 
administration. The PRO-S/E is but one part of the information^ training, 
and technical assistance provision offered to districts, educator associa- 
tions, and individual educators as part of AEL^s regional laboratory ser- 
vices. These two constaints led PRO-S/E developers to create a mutually 
beneficial opportunity in training university faculty as PRO-S/E associates. 
This section of the paper is organized around key questions significant to 
understanding PRO-S/E associate training. 

How do faculty and colleges of education benefit from becoming PRO-S/E 
associates? Traditionally, individual faculty and colleges of education must 
demonstrate their commitments to serve the education community, school dis- 
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tricts which provide them with undergraduate and graduate students, research 
access, placement for preservice teachfir field experiences, and career oppor- 
tunities for teacher education program gradua\:es. Individual faculty members 
are often interested in consulting opportunities to supplement university 
salaries. In Tennessee, university faculty involved in teacher preparation 
are required to spend at least six days per year in provision of direct ser- 
vice to school districts in the stale. Through PRO-S/E associate training, 
individual and college of ec!ucation service mandates can be facilitated, 
while districts receive school effectiveness diagnosis upon which to plan 
school improvements. Beyond mandates and financial gain ambitions, training 
in the PRO-S/E can expand an indi^ndual professor^'s knowledge of the school 
effectiveness literature and establish a framework related to the eleven 
PRO~S/E school effectiveness characteristics upon which college of education 
faculty may build staff development/technical assistance programs appropriate 
for many school districts. 

What is the role of a PRO-S/E associate? Following training, an individ- 
ual faculty member, as a PRO-S/E associate, may contact school superinten- 
dents to introduce the PRO-S/E and market his/her services as a PRO-S/E ad- 
ministrator. Following contract negotiations between the associate and the 
superintendent, the associate contacts AEL with details essential for AEL^'s 
role in an acknini strati on. The AEL role includes provision of instruments, 
coded answer sheets for the instruments, data analysis, and provision of data 
analysis output for use by the associate in preparing the final report. The 
associate is then responsible for arranging and conducting interviews, dis- 
tribution of survey instruments and answer sheets to the schools, handling 
district concerns and questions, submitting the answer sheets to AEL, inter- 
preting the output from AEL, writing the final report, and meeting with the 
district administrators to present the results, discuss follow-up staff 
development, and respond to questions. 

Why would a faculty member want to handle all the responsibilities of a 
PRO-S/E ackninistration? Besides the commitment to service, previously cited, 
a PRO-S/E associate may receive extra compensation for adninistering the PRO- 
S/E, assuming his/her university agrees to such an arrangement. The price of 
the PRO-S/E administration to the district is fixed by AEL on the basis of 
the number of schools and the number of teacher, student, and staff surveys 
analyzed. Additional reimbursement for travel costs are also included in the 
price the district pays. Actual expenditures of an administration may vary 
depending on the number of faculty and/or graduate assistants involved. The 
associate-'s university may, and are encouraged to, use PRO-S/E funds to pay 
faculty extra compensation for their involvement in the PRO-S/E ackninistra- 
tion. AEL receives a stipend from the associate^'s university the first time 
a PRO-S/E is conducted. This stipend reimburses AEL for training costs. 
Each time a PRO-S/E is conducted, AEL receives a per-subject fee for answer 
sheets and data analysis costs. While the associate may incur overhead costs 
for use of university services and facilities, there should be an adequate 
amount left to compensate those involved. 

What are the objectives and purposes of a PRO-S/E training session? The 
primary objective of the full-day session is to train associates to handle 
PRO-S/E contacts, contracts, administration, data interpretation, report 
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writing, follow-up presentation, and school improvement planning with a min- 
imum of assistance from ftEL. AEL staff leading the training session organize 
activities to accocn>l ish the following six purposes: 

!• Acquaint associates with PRO-S/E developers and establish lapport for 
future coGxnuni cat ions, 

2. Describe administrative procedures, highlighting success hints and 
problems to avoid, 

3. Train associates in PRO-S/E interviewing, survey acknlnistration, and 
data interpretation^ 

4. Encourage consultant behavior role-taking by associates, many of whom 
have not marketed their skills independently, 

5. Provide discussion opportunities to address associate questions and 
concerns, and 

6. Overview the training manual contents and uses. 

What does the manual provide to associates? The PRO-S/E manual is the 
primary resource an associate needs after training. Arranged in sections 
corresponding to the step-by-step acknlnistration of the PRO-S/E, the manual 
contains a sample contact letter, a sample contract, a.n overview of the lit- 
erature used in the development of the PRO-S/E, effectiveness character- 
istic definitions, an abstract describing the profile, haru-copy from which 
surveys may be duplicated, all interview schedules, data analysis examples, 
hints for writing the final report, and an Ackninistrator^s Checklist for the 
process based on a suggested timeframe. 

AEL staff have conducted three training sessions to date in which higher 
education faculty from Kentucky, Tennessee, and West Virginia have become 
PRO-S/E associates. During training and in subsequent contacts with associ- 
ates, AEL staff assess concerns over PRO-S/E administration and provides con- 
sultation as associates arrange, administer, interpret, report, and follow-up 
the PRO-S/E. We believe the training and follow-up, plus AEL^s control of 
data analysis and price, ensure reliable diagnosis of school effectiveness 
for contracting districts and a productive and profitable relationship with 
AEL and school districts for the PRO-S/E associates. 



FOLLOW-UP WITH SCHOOL DISTRICTS AFTER THE USE OF THE PRO-S/E - Jane Hange 

Whether coordinated by AEL or an AEL-trained PRO-S/E associate, one of the 
most important steps of the process is the follow-up meeting. This overview 
of PRO-S/E results begins with an explanation of the purpose of the PRO-S/E 
as diasiiosis for go&l setting and staff development planning and can lead to 
a multi-year ccT'erative arrangement for technical assistance. 

The fc^lr .*^tiiij is convened by the district superintendent approx- 
Imat'cly 10 • j.:^^r he/she receives the PRO-S/E data displays and 
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suranary report. The two to three hour session, conducted by AEL staff or 
the PRO-S/E associate, is usually attended by the superintendent, central 
office staff, and building principals whose schools were assessed. The pur- 
poses of the meeting are to: 

1* Reacquaint district personnel with the purposes of the PRO-S/E and the 
data sources, 

2. Review data display sets for different types of summaries (total 
schools, total elementary, total middle, total secondary, and for 
each school ), 

3. Provide brief interpretation of data displays for most significant 
characteristics of total district data, 

4. Emphasize perceptual basis for the data and PRO-S/E as a "snapshot" 
of school effectiveness to be used as baseline data for improvement 
planning, 

5. Describe a variety of R & D-based instructional improvement options 
keyer* to the 11 PRO-S/E school effectiveness characteristics, 

6. Respond to -particular questions in large group and one-on-one 
consultation., and 

7. Offer follow-up correspondence and phone consultation to individual 
principals as they prepare to present results to faculty and plan 
schoo 1 i mprovemen ts . 

AEL offers School Excellence Site (SES) status to districts within AEL^s 
service region (Kentucky, Tennessee, Virginia, and West Virginia) following 
their participation in the PRO-S/E process. The cooperative arrangement, a 
non-legal contract between AEL and the district, inaugurates SES status and 
outlines AEL technical assistance to be provided over a three to five year 
period. Staff development planning, information acquisition and electronic 
searches. Resource Center loaned materials, toll-free phone consultation 
(including Techline), and invitations to participate in training-for-trainers 
are extended to SES districts. Based upon the district's PRO-S/E results 
and district priorities, AEL may assist district personnel in implementing an 
R & D-based program or in becoming trained to lead faculty workshops in one 
of 13 School Excellence Workshop packages developed to address many of the 11 
characteristics of the PRO-S/E. To the variety of products and services of- 
fered at no or low cost to the districts, AEL membership is added, providing 
practitioners with AEL^s quarterly newsletter. The Link , and myriad other 
products and publications from AEL and the Lab-Center network. In c^xchange 
for this assistance, the district agrees to allow AEL to evaluate over time 
the effectiveness and impact these services have had on district teaching 
and learning practices and outcomes. 

Al Osborne, Superintendent of Frankfort Independent Schools (KY), has 
previously described the working relationship between AEL and a school dis- 
trict that can follow PRO-S/E administration. Another case in ooint is the 
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Harrison County School District In Clarksburg, West Virginia, where the PRO- 
S/E diagnosis of school effectiveness pointed toward parent involvement and 
use of instructional time as characteristics most in need of improvement* 
AEL, district administrators, and faculty are working together in the follow- 
ing ways: 

1. Development of a five-year educational management plan utilizing PRO- 
S/E results for the district with similar five- and one-year plans 
developed by school principals, 

2. Technical assistance in bringing on-line a computer (card read) school 
lunch reporting system, developed by a Harrison County principal, to 
free up teacher instructional time, 

3. Consultations with individual principals and central office staff to 
interpret PRO-S/E results and plan appropriate school improvements, 

4. Invitations to participate in AEL training-for-tralner sessions on 
"Parent Involvement" and "Community Support Through Public Relations", 

5. Participation by the Harrison County Superintendent, Robert Kittle, in 
the AEL Study Group Annual Conference, and 

6. Technical assistance in district-wide computer tracking of student 
progress. 

With 40 schools and 1,043 professional personnel, Harrison County and AEL 
will undoubtedly identify and capitalize on numerous opportunities for im- 
provement as an AEL SES. But with more than 40 or fewer than four schools, 
any district can benefit from the PRO-S/E, a diagnosis of school effective- 
ness often followed by mutually planned prescriptions for school improvement. 
The PRO-S/E becomes a focal point for discussion and action for the adminis- 
trators, board members, and faculty in a school district. 



References (these are available from AEL): 

Sanders, J., Shively, J., and Machesney, J. D. (1984). School effective- 
ness: Profile of school excellence. Paper presented at AERA Annual 
Meeting, New Orleans, LA. ED 249 228 

Sanders, J., Barnette, J., and Vanco, P. (1986). A system for measuring 
school effectiveness: The PRO-S/E. Paper presented at AERA Annual 
Meeting, San Francisco, CA. ED 269 471 
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ATTACHMENT A 



DEFINITIONS 



1. Needs Basis : The extent to which gchool personnel use an in-place 
system for identifying, teaching, evaluating, and remediating student 
learning needs. 

2. Objectives : The degree to which school personnel prescribe and 
communicate to students relevant and attainable objectives for each 
academic course* 

3. Expectations ; The degree to which school personnel communicate 
clearly to each student the belief (the expectation) that each can 
and will succeed in attaining prescribed academic objectives. 

Roles & Responsibilities : The degree to which school personnel roles 
are defined and understood and the degree to which school personnel 
prepare each student to assume an appropriate level of responsibility 
for learning, to cooperate with others, and to participate in a broad 
range of academic and non**academic activities. 

5» Conditions & Resources : The degree to which school personnel provide 
students exemplary conditions of learning — that is, grouping students 
appropriately; presenting and modeling information and skills in an 
interactive way that properly motivates students; and using excellent 
instructional materials to assure maximum student participation and 
success. 

6. Instructional Time b Task Orientation : The degree to which school 
personnel provide students maximum instructional time during class 
periods and assure that students attend to and successfully engage 
in appropriate academic tasks during class time. 

7» Use of Assessment : The degree to which school personnel use assess- 
ment data as the basis for informing students of their academic 
progress and informing teachers of their students' remediation needs. 

8» Rewards & Reinforcement : The degree to which school personnel use an 
in-place system of reinforcement that recognizes the accomplishments 
and achievements of student end staff. 

9» Code of Behavior : The degree to which school personnel communicate 
clearly and enforce equitably rules, struvtture, iToutines, and conse- 
quences governing student behavior. 

10* School Climate ; The degree to which school personnel create and 
model a collegial environment in which students receive and return 
to those around them a sense of caring, personal concern, interest, 
respect, commitment, and support for persons, property, and ideas. 

11 • Parental Support & Involvement : The degree to which s .hool personnel 
have established procedures that encourage meaningful parental and 
community interest, involvement, and support in students* academic 
progress. 
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ATTACHMENT B 



INSTRUMENTS 

District Data Form : Used to collect data from the superintendent on the 
district's educational philosophy, descriptior of the schools in the 
district, results of any district surveys conducted, district 
budget, rank in state relative to per-pupil expenditure, percent of 
graduates attending postsecondary institutions, standardized test 
results, and community information. 

Superintendent Interviev : Used to collect data from the superintendent on 
his/her professional background, perceptions of the greatest local 
educational needs, programs for improving students* basic skills, 
communication of expectations, encouragement of staff professional 
initiative, evidence of district successes, communication of 
standards/code of conduct, teachers* perceptions of professional 
development, staff roles and goals, staff hiring and orientation 
procedures, monitoring/evaluation of principals, use of research, 
perceptions of school climate, perceptions of community/parental 
involvement, and most serious problems to be faced by the 
superintendent and principals during the next five years. 

School Data Form ; Used to collect data about each school on enrollment 
statistics for past three yers; average daily attendance; 
pupil-teacher ratio; current innovative programs; any recent survey 
results; student followup information; newsletters; policy 
statements on communication of achievement data, use of achievement 
data by teachers and counselors, and school educational philosophy 
including teacher code and student code; and, if a secondary school, 
a listing of courses available. 

Principal Interviev : Used to collect data from each school principal on 
his/her professional background; school academic goals and 
communication of goals to faculty, students, and parents; 
perceptions of effectiveness of basic skills achievement programs; 
encouragement of staff initiative; use of fiscal resources; 
attendance policies; procedures for monitoring instructional time 
and academic progress; recognition of student academic achievement; 
rules of classroom behavior; staff roles; evaluation of teachers; 
perceptions of school climate; encouragement of parental and 
community support; and problems principals will face in the next 
five years. 

School Rating Form : Used by central office staff to rate each school on 
the 11 PRO-S/E characteristics. 

Teacher Questionnaire : A 50 percent sample of nonitinerant teachers from 
each school is selected at random to complete a questionnaire that 
measures perceptions of district and school status on the 11 PRO-S/E 
characteristics. 
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Student QxiestiotiRaire : A 10 percent sample of students from grades 6-12 
from each school is selected at random to complete a questionnaire 
that measures perceptions of status on the 11 PRO-S/E 
characteristics. 

Test Analysis ; There is an eighth instrument used to conduct a study of 
the schools' achievement test scores and students' socioeconomic 
status. Sometimes school districts have such studies on file, 
sometimes not. This study accepts Ron Edmonds' finding that in 
effective schools, standardized test scores indicate that learning 
opportuniti^^s and achievement are similar for students from various 
backgrounds and SES. 
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ATTACHMENT C 

* PRD-S/E Analysis, Copyright 19B6, The Appalachia Educational Laboratory, Inc. 

District: Calaveras County Schools Date: 11/22/86 Broup: Total, All Schools Nuiber of Schools- 13 

Tible I. A Means and Standard Deviations for Each Characteristic by Respondent Broup 

Students Teachers Staff Rtgs. 

n= 916 n= 247 n= 142 



Characteristic 


Hean 


SO 


Hean 


SD 


Hean 


SD 


Needs Basis 


16.75 


3.18 


14.47 


2.53 


12.82 


2.91 


Objectives 


15. 5B 


4.01 


17.36 


2. IB 


13.51 


2.51 


Expectations 


16.25 


4.06 


14.19 


3.43 


13.32 


2.39 


Roles/Responsibilities 


15.30 


3.49 


1/..51 


2.69 


13.96 


2.78 


Conditions/Resources 


14.43 


4.54 


15.88 


2.51 


13.96 


1.90 


Instr. Tiie/Task Orient. 


12.99 


4.57 


13.74 


2.27 


13.82 


2.86 


Use of Assessment 


14.10 


3.95 


13.25 


3. 86 


13.00 


2.13 


ReHards/Reintorcessnt 


14.65 


4.20 


15.19 


2.35 


13.43 


2.92 


Code of Behavior 


16.48 


3.50 


15.55 


2.91 


13.57 


3.16 


School Cliiate 


14.88 


4.07 


15.92 


2.61 


13.40 


2.36 


Parental Supp'ort/lnvolveaent 


14.36 


4.88 


12.71 


2.54 


13.68 


2.62 
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PRO-S/E; Total, fill Schools 




Superior 



I flcceptaUe 



' Marginal 



llnacceptaUe 
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ATTACHMENT E 



< PRO-S/E Analysis, Copyright 1986, The Appalachia Educational Laboratory, Inc. 

District: Calaveras County Schools Date: 11/22/86 Group: Total, All Schools Nuiber of Schools^ 15 



Table VILA Teacher Survey Responses, n= 247 Page 2 

Characteristics Ranked High to Lo)«, Iteis Hithin Characteristics Ranked High to Lom 



5. Rewards/Reinforceient Hean- 15.19 (Acceptable) 


n 


SC 


Rating 


1. Frequency praise in class used to reinforce students 


247 


0.98 


Superior 


2. Reinforceient of students a topic of faculty leeting/inservice 


246 


0.95 


Superior 


3. Special privileges used to reinforce students 


247 


0.75 


Acceptable 


4. Stud°nts are sufficiently reinforced for acadeiic progress 


246 


0.75 


Acceptable 


5. Schooi-Nide eiphasis on providing systeiatic reinforceient 


244 


0.73 


Acceptable 


6. Frequency notes sent to parents to reinforce students 


247 


0.70 


Acceptable 


7. Noiination for Honor Roll/in-school recognition used to reinforce students 


246 


0.70 


Acceptable 


B. Special iention in local/school neHspaper used to reinforce students 


246 


0.51 


Marginal 


6. Roles/Responsibilities Hean= 14.51 (Acceptable) 


n 


SC 


Rating 


K Roles of school personnel defined in forial docuients 


244 


0.80 


Acceptable 


2. Teacher understanding of role of principal 


247 


0.80 


Acceptable 


3. Students understand their roles/responsibilities 


247 


0.77 


Acceptable 


4. Students adequately prepared for their futures 


246 


0.75 


Acceptable 


5. Lack of reservations of role/responsibilities as a teacher in this school 


247 


0.72 


Acceptable 


6. Students participate in balance of acadeiic and extracurricular activities 


246 


0.72 


Acceptable 


7. Teacher understanding of role of superintendent 


247 


0.64 


Marginal 


B. Teacher understanding of role of school board 


247 


0.62 


Marginal 


7. Needs Basis nean= 14.47 (Acceptable) 


n 


SC 


Rating 


1. Use of student listake patterns for diagnosis 


245 


0.83 


Acceptable 


2. Rational basis for subject latter decisions 


247 


0.81 


Acceptable 


3. Effectiveness of diagnostic procedures 


247 


0.80 


Acceptable 


4. Rational basis ^or subject latter per grade level decisions 


245 


0.77 


Acceptable 


S. Usefulness of resources for diagnosing student learning needs 


246 


0.72 


Acceptable 


6. Use of information froi previous teachers for diagnosis 


Tit 

246 


ft 1ft 

0.70 


Acceptable 


7. Effectiveness of resediation procedures 


246 


0.69 


Marginal 


B. Use of information fron state testing prograi for diagnosis 


247 


0.68 


Marginal 


9. Frequency of adiinistration of diagnostic tests 


241 


0.51 


Marginal 


8. Expectations Kean= 14.19 (Acceptable) 


n 


SC 


Rating 


1. Coiiunication of expectations to students 


247 


0.80 


Acceptable 


2. Teachers have had preservice/inservice on student expectations 


244 


0.72 


Acceptable 


3. Students can aaster instructional objectives of courses taught 


246 


0.70 


Acceptable 


4. Students can learn if given proper instruction/reiediation 


246 


0.62 


Marginal 



SC= Scale Coefficient (Range: Negative/False/LoN to 1= Positive/True/High, Group Mean Response on Itei) 
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ATTACHMENT F 



PRG-S/E Analysis, Copyriqht 19B6, The Appalachia Educational Laboratory, Inc. 

District: Calaveras Couiity Schools Date: 11/22/86 Broup: All Schools Nuibsr of Schools= 13 

Table IV.A.TK Keans for Each Characteristic by School and Respondent Broup, Instr. Tiie/Task Orient. 



Students teachers Staff Rtgs. 



School 


Type 


n 


!1ean 


n 


Hean 


n 


Kean 


1. HuH Potter Ele.en 


E 


iB 


14.14 


16 


13.46 


11 


14.95 


2. Injun Joe Eleienta 


E 


46 


13.83 


21 


12.32 


10 


12.00 


3. Becky Thatcher Ele 


E 


55 


13.77 


12 


11.11 


11 


11.92 


4. Aunt Polly Eleient 


E 


18 


11.19 


11 


14.09 


1! 


13.54 


5. Puddn'head Wilson 


E 


43 


14.23 


14 


14.63 


11 


13.94 


6. Toi Sawyer Eleient 


E 


44 


11.97 


13 


14.15 


n 


14.34 


7. Huck Finn Eleienta 


E 


34 


13.12 


10 


14.60 


10 


14.89 


G. Horace Bixby Eleie 


E 


68 


13.40 


18 


12.93 


11 


13.74 


9. Bixby's Cub Hiddle 


H 


60 


11.59 


19 


14.17 


u 


13.13 


10. Extraordinary ThIo 


n 


60 


14.00 


19 


14.53 


11 


14.95 


11. Charles Hebster Hi 


S 


136 


16.33 


26 


13.23 


11 


14.55 


12. Hark Twain High 


S 


190 


10.80 


46 


14.80 


12 


13.52 


13. Hannibal High 


S 


94 


11.07 


22 


13.39 


11 


14.14 



Correlations of Group Keans Across Schools (Unit of Analysis), and One-Tailed Significance Tests of r > 0 



Groups 


n 


r 


Sign. 


Student - Teacher 


13 


- 0.32 


N.S. 


Student - Staff 


13 


0.16 


N.S. 


Teacher - Staff 


13 


0.64 


N.S. 
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